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© Vehicle recognition apparatus. 



© An apparatus for recognizing contours of a preceding vehicle from road images inputted by the use of 
vehicle-mounted cameras for picking up an object in front of the apparatus-mounted vehicle and measuring a 
distance to the recognized preceding vehicle, comprising image inputting video cameras, AID converters, image 
memories, image processing MPU, a display, an output interface, and the like. 
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BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

5 The present invention relates to an apparatus for recognizing a preceding vehicle by the use of an 
image processing technology and measuring a distance to the recognized vehicle. 

2. Description of the Prior Art 

w Heretofore, as described in Japanese Laid-Open Patent HEI 1-281600, some of such apparatus for 
recognizing a preceding vehicle have traced edges extracted from images to extract a preceding vehicle 
existing region. 

However, such conventional vehicle recognition apparatus have had a problem that where vehicle edges 
extracted by an edge extraction processing has a disconnection, the region cannot be exactly extracted. 
75 The distance measuring by stereo vision has had also a problem that there is developed a difference 
between vehicle shapes observed by the right-hand and left-hand cameras, so that a correlation between 
two images is hardly established. 

SUMMARY OF THE INVENTION 

20 

A first object of the present invention is to provide an apparatus for recognizing a preceding vehicle 
from road images inputted by the use of video cameras and measuring a distance to the recognized 
preceding vehicle. 

A second object of the present invention is to provide an apparatus for extracting regions in which the 

25 preceding vehicle seems to exist on the basis of the distribution of edges scattered in the images or of the 
right/left symmetricalness of the preceding vehicle. 

A third object of the present invention is to provide an apparatus for setting automatically an initial 
model of Active Contour Models. 

A fourth object of the present invention is to provide an apparatus for extracting contours of the 

30 preceding vehicle by the use of the technique of the Active Contour Models. 

A fifth object of the present invention is to provide an apparatus for measuring the distance to the 
preceding vehicle by the use of the contours of the preceding vehicle extracted from the images. 

In order to achieve the above-mentioned objects, the present invention includes stereo cameras which 
are mounted on a vehicle to pick up road scenes in front of the vehicle, A/D converters for A/D converting 

35 analog image signals inputted from the cameras, image memories for storing road images digitalized by the 
A/D converters, ROM, RAM as a work region for accumulating data or programs, an image processing 
microprocessor for processing the road images stored in the image memories, a display for displaying 
processing results, a display controller for controlling the display, and an output interface for implementing 
other application functions. The image processing microprocessor consists of a vehicle recognition section 

40 for recognizing the preceding vehicle from the inputted images and an intervehicle distance measuring 
section for measuring the distance to the recognized preceding vehicle to perform processing. The vehicle 
recognition section comprises an edge extraction subsection for applying differential processing to the road 
images stored in the image memories to extract edges, a traffic lane region extraction subsection for 
extracting traffic lane regions from the road images stored in the image memories, an edge searching 

45 subsection for extracting vehicle candidate regions from the road images stored in the image memories, a 
symmetrical region extraction subsection for extracting right/left symmetrical regions from the vehicle 
candidate regions searched from the edge searching subsection to limit further the vehicle candidate 
regions, an initial model setting subsection for setting models for the symmetrical regions extracted by the 
symmetrical region extraction subsection, and a vehicle contour extraction subsection for extracting 

so contours of the preceding vehicle on the basis of the symmetrical regions extracted by the symmetrical 
region extraction subsection and of the information on shapes of models set by the initial model setting 
means. The intervehicle distance measuring section comprises a shift pattern preparation subsection for 
preparing a pattern performing a shift operation with respect to a reference pattern, a disparity extraction 
subsection for shifting the shift pattern prepared by the shift pattern preparation subsection with respect to 

55 the reference pattern and establishing a correlation between both the patterns so as to extract a disparity, 
and an intervehicle distance calculation subsection for calculating the intervehicle distance on the basis of 
the disparity extracted by the disparity extraction subsection and of camera positional information. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram showing a basic composition of an embodiment of the present invention. 

Fig. 2 is a view showing an example of installation of cameras on a vehicle. 
5 Fig. 3 is a flowchart showing an operation of the embodiment of the present invention. 

Figs. 4 (a) and 4 (b) are views showing examples of stereo images. 

Fig. 5 is a flowchart showing an operation of the vehicle recognition processing. 

Fig. 6 is a view showing results obtained by extracting edges from an inputted image. 
v Fig. 7 is a flowchart showing an operation of the vehicle region extraction processing. 
10 Fig. 8 is a typical view showing a processing for extracting white line contours. 

Fig. 9 is a view showing results obtained by extracting the white line contours. 

Fig. 10 is a view showing results obtained by extracting traffic lane regions. 

Fig. 11 is a view showing a method of extracting adjacent traffic lane regions. 

Figs. 12 (a) and 12 (b) are typical views showing a concept of a processing for extracting the lower 
15 ends of vehicle candidate regions in the edge searching processing. 

Figs. 13 (a) and 13 (b) are typical views showing a concept of a processing for extracting the right/left 
ends of vehicle candidate regions in the edge searching processing. 

Fig. 14 is a view showing results obtained by extracting the vehicle candidate regions by the edge 
searching processing. 

20 Fig. 15 is a view showing a processing range in the symmetric region extraction processing. 

Fig. 16 is a typical view showing a processing concept of the symmetric region extraction processing. 

Figs. 17 (a) and 17 (b) are views showing results of the symmetric region extraction processing. 

Fig. 18 is a flowchart showing an operation of the symmetric region extraction processing. 

Fig. 19 is a view showing an initial model of Active Contour Models. 
25 Fig. 20 is a view showing a setting state of the Active Contour Models. 

Fig. 21 is a view showing a method of minimizing an energy of the Active Contour Models. 

Fig. 22 is a flowchart showing an operation of the contour extraction processing by the Active Contour 
Models technique. 

Fig. 23 is a view showing results obtained by extracting vehicle contours by the contour extraction 
30 processing. 

Fig. 24 is a flowchart showing an operation of the intervehicle distance measuring processing. 
Figs. 25 (a) and 25 (b) are views showing results obtained by extracting vehicle contours from Figs. 4 
(a) and 4 (b). 

Figs. 26 (a) and 26 (b) are views showing contour models of the contour extraction results shown in 
35 Figs. 25 (a) and 25 (b). 

Fig. 27 is a view showing a shift pattern for disparity extraction. 
Fig. 28 is a view showing a shift operation for disparity extraction. 
Fig. 29 shows an example of the output to the display. 

40 DESCRIPTION OF THE PREFERRED EMBODIMENTS 

With reference to drawings, an embodiment of the present invention will be explained hereinafter. 
Figs. 1-A and 1-B are block diagrams showing a basic composition of an embodiment of the present 
invention. The numerals 1a and 1b indicate a pair of cameras for picking up an object in front of a vehicle 

45 and installed on the front side of the vehicle in a manner not to prevent the field view of a driver as shown 
in Fig. 2. The numerals 2a and 2b indicate A/D converters each for digitalizing analog image signals 
inputted from the video cameras 1 and 2. The numerals 3a and 3b indicate image memories for storing 
road images digitalized by the A/D converters 1 and 2. The numeral 4 indicates a ROM (Read Only 
Memory) for loading programs describing processing contents of the present invention; and the numeral 5 

so indicates a RAM (Random Access Memory) as a work memory for storing programs or data. The numeral 6 
indicates an image processing MPU (Micro Processor Unit), whose processing contents are broadly 
classified into a vehicle recognition section 6a for extracting contours of a preceding vehicle, and an 
intervehicle distance measuring section 6b for measuring a distance to the preceding vehicle, as shown in 
Fig. 1-B. The vehicle recognition section 6a further comprises an edge extraction subsection 61 for 

55 extracting edges from the road images, a traffic lane region extraction subsection 62 for extracting traffic 
lane regions, an edge searching subsection 63 for extracting vehicle candidate regions, a symmetric region 
extraction subsection 64 for extracting right/left symmetric regions in the vehicle candidate regions 
extracted by the edge searching subsection, an initial model setting subsection 65 for setting an initial 
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model of Active Contour Models, and a vehicle contour extraction subsection 66 for extracting vehicle 
contours by the use of the Active Contour Models technique. The intervehicle distance measuring section 
6b comprises a shift pattern preparation subsection 67 for preparing a pattern performing a shift operation 
with respect to a reference pattern, a disparity extraction subsection 68 for determining a disparity between 

5 the reference pattern and the shift pattern, and an intervehicle distance calculation subsection 69 for 
calculating the distance to the preceding vehicle. The numeral 7 indicates a display controller for 
performing various settings on a display 8 for displaying processing results. The numeral 9 indicates an 
output interface for implementing various application functions using intervehicle information calculated by 
this apparatus, such as an autocruise for cruising the vehicle with the distance to the preceding vehicle kept 

w constant, or an intervehicle distance alarm device for alarming the driver if the distance to the preceding 
vehicle becomes a certain value or less. 

Fig. 3 is a flowchart describing a processing procedure in the embodiment of the present invention. 
According to the flowchart, the processing contents will be explained hereinafter. 

First, at the step 1000, various initializations are performed to clear registers or counters so as to repeat 

75 the following processing. 

Then at the step 2000. road images are picked up. First, the road images are inputted as analog image 
signals by the use of a pair of the stereo cameras 1a and 1b. Then, these analog image signals are 
digitalized in &-bit gradation value by the A/D converters 2a and 2b, and then stored in the image memories 
3a and 3b. Figs. 4a and 4b show an example of stereo road images thus picked up. The image shown in 

20 Fig. 4a is an input image from the video camera 1 , while that shown in Fig. 4b is an input image from the 
video camera 2. 

Then at the step 3000 of Fig. 3, the preceding vehicle is recognized from the road images picked up at 
the step 2000. In the preceding vehicle recognition processing, a pair of stereo images are subject to the 
same processing, respectively, so that only the processing for the input image from the video camera 1 will 

25 be described hereinafter. / 
Fig. 5 shows a flow of a series of processing for the vehicle recognition processing. 
First, at the step 3100 of Fig. 5, a differential proccesing is applied to the road images picked up at the 
step 2000 of Fig. 3 to extract edges. A 3 X 3 sobel filter is used to extract edges, and differential intensities 
obtained for each pixel are stored in the image memories 3a and 3b. Fig. 6 shows results obtained by 

30 making two values the differential intensity E (x, y) by the use of the threshold Eth. The threshold Eth varies 
with the dyanamic range and the like of the video camera 1, and a range in which nine 8-bit gradation 
values are added or subtracted is within a range from 0 to 1 530 , so that it is preferable that the threshold 
Eth is set within a range from 80 to 120. Points shown with black in Fig. 6 are pixels having a differential 
intensity exceeding the threshold Eth, which pixels will be called edge pixels hereinafter. 

35 Then at the step 3200 of Fig. 5, traffic lane regions are extracted from the road images. The flowchart 
of Fig. 7 shows a series of processing for the vehicle region extraction processing. In this case, in order to 
improve a contour extraction accuracy of white lines, the processing region is limited to the lower half 
portion of images. 

At the step 3201 of Fig. 7, the contour of the left-hand white line painted on a road is extracted. First, as 
40 shown in Fig. 8, pixels are scanned from the center line of respective scanning lines in the left direction. 
Then, an initial position at which the differential intensity E (x, y) of each pixel determined at the step 31 00 
of Fig. 5 exceeds the threshold Eth is taken as the contour point of the left-hand white line in the scanning 
tine. 

Then at the step 3202 of Fig. 7, the contour of the right-hand white line is extracted. In a similar manner 
45 to the left-hand white line, the pixels of the contour of the right-hand white line are scanned from the center 
line of respective scanning lines of Fig. 8 in the right direction, and an initial position at which the differential 
intensity E (x, y) exceeds the threshold Eth is extracted as the contour point of the right-hand white line in 
the scanning line. Fig. 9 shows results obtained by extracting the contour point row of the right/left-hand 
white lines. Pixels shown with black point are the contour points of the right/left-hand white lines, 
so Then at the step 3203, the contour point row of the left-hand white line extracted at the step 3201 is 
approximated in straight line. 

In a similar manner, at the step 3204, the contour point row of the right-hand white line extracted at the 
step 3202 is approximated in straight line. The technique of the Hough transformation (U.S. Pat. No. 
3,069,654 (1962)) is used for the straight-line approximation of point rows performed at the steps 3203 and 
55 3204. 

Further at the step 3205, a region formed with the left-hand white line approximated line, the right-hand 
white line approximated line, the image lower end, the image left end and the image right end is extracted 
as a traffic lane region in which the apparatus-mounted vehicle cruises. Fig. 10 shows results obtained by 
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extracting the traffic lane region in which the apparatus-mounted vehicle cruises. The region surrounded by 
the VLBA is the extracted traffic lane region. 

Further at the step 3206, adjacent right/left traffic lane regions adjacent to the apparatus-mounted 
vehicle traffic lane region extracted at the step 3205 are approximately determined. In this case, as shown 

5 in Fig. 11, the triangle VCD obtawined by extending double rightward/leftward the length of the base LR of 
the triangle VLR indicating the apparatus-mounted vehicle traffic lane region is extracted as the road image 
including the adjacent traffic lane regions. 

Then at the step 3300 of Fig. 5, the distribution of edges scattered in the traffic lane region extracted at 
the step 3200 is checked to determine the preceding vehicle existing candidate region. 

io Figs. 12 and 13 show typical views showing the concept of the processing. First, as shown in Fig. 12b, 
a processing region is limited to the apparatus-mounted vehicle traffic lane region extracted at the step 
3205 of Fig. 7. Then, in the processing region, the number of pixels having an edge intensity exceeding the 
threshold Eth (called edge pixels) is counted for each scanning line to prepare a histogram as shown in Fig. 
12a. At the same time, an average coordinate position of these edge pixels in the scanning line direction is 

15 determined and taken as the gravity center Gx of the vehicle candidate region in the scanning line direction. 
The axis of ordinate of the histogram represents respective scanning lines, while the axis of abscissa 
represents the number of edges pixels. Then, a threshold Bth is set with respect the number of edges 
pixels. The experiment performed using various images resulted in that the threshold Bth is preferably set 
to about 40. Among scanning lines in which the number of eges pixels exceeds the Bth, the scanning line 

20 positioned at the lowest position is extracted as the lower end of the vehicle candidate region. In Fig. 12a, 
the scanning line shown with B indicates the lower end of the vehicle candidate region. Where such 
scanning line satisfying the above-mentioned conditions is not extracted, the preceding vehicle is judged 
not to exist. 

On the contrary, where the lower end of the vehicle candidate region is extracted, the right/left ends of 

25 the vehicle candidate region are extracted. Figs. 13a and 13b show typical views showing the concept of 
the processing. First, as shown in Fig. 13a, a processing region is limited to the road region including 
adjacent traffic lanes extracted at the step 3206 of Fg. 7 and to the region surrounded by the lower end of 
the preceding vehicle candidate region extracted by the preceding processing and the right/left ends of the 
image. The reason why such processing range is set is that the adjacent traffic lanes are observed at all 

30 times for the traffic lane change of the preceding vehicle or the interruption by another vehicle. Within the 
processing range, the number of edge pixels is counted for each vertical pixel column perpendicular to the 
scanning lines to prepare a histogram as shown in Fig. 13b. The axis of abscissa of the histogram 
represents the lateral coordinates of the image, while the axis of ordinate represents the number of edge 
pixels. In this case, as apparent from Figs. 13a and 13b, it is understood that in the region outside the 

35 candidate region in which the preceding vehicle exists, compared to the inside of the vehicle candidate 
region, the frequency of the histogram rapidly decreases (P) and its dispersion becomes small (Q). Then, 
according to the following procedure, the right/left side ends of the vehicle candidate region are extracted. 
First, small windows having an interval width Sw are provided with respect to the axis of abscissa of the 
histogram. Then, while calculating the mean value Emean and dispersion value Esigma of the edge 

40 frequency in the small windows, the windows are caused to be shifted from the gravity center position Gx of 
the vehicle candidate region to the outside rightward/leftward. Then, initital positions at which the Emean 
becomes the threshold Mth or less and the Esigma becomes the threshold Sth or less are extracted as the 
left end and the right end, respectively, of the vehicle candidate region. Preferably, the small window 
interval width Sw is set to abount 20 pixels; the threshold Mth for the Emean, to about 15 pixels; and the 

45 threshold Sth for the Esigma, to about 17 pixels. Fig. 14 shows results obtained by extracting the vehicle 
candidate region by the above-mentioned processing. 

At the step 3400 of Fig. 5, symmetrical regions are extracted. Generally, the preceding vehicle 
displayed on an iamge screen shows a substantially symmetrical shape with a segment perpendicular to 
sanning lines taken as a symmetrical axis. Thus, within the preceding vehicle candidate region defined at 

50 the step 3300, the symmetrical region is extracted, thereby further limiting the preceding vehicle existing 
region- 
According to the typical views shown in Figs. 15 through 17, the outline of these processings will be 
explained. First, as shown in Fig. 15, the processing range in which the symmetrical region is extracted is 
limited to the region within the preceding vehicle candidate region defined at the step 3300. Then, within 

55 the processing region, a symmetrical axis perpendicular to sanning lines is determined. For example, as 
shown in Fig. 16a, assuming that a symmetrical point with respect to the point A on the same scanning line 
is the point B, their symmetrical axis S2 passes through the mid point between the points A and B. 
Although no view is shown, on the same scanning line for the points A and B, there can be another pair of 



5 



EP0 626 655 A2 



symmetrical points (thus, there can be another symmetrical axis). In a similar manner, assuming that a 
symmetrical point with respect to the point G is the point H, their symmetrical axis S1 passes through the 
mid point between the points G and H. The position of such symmetrical axis is calculated for each pair of 
edge points distributed in the processing region to prepare a histogram as shown in Fig. 16b. Then, the 

5 position indicating the peak of the histogram is extracted as a symmetrical axis. The symmetrical region is 
extracted by searching edge points becoming a pair with respect to the symmetrical axis. 

Figs. 18-A and 18-B show a series of flow of the processing. First, through the processing performed at 
the steps 3401 through 3413, the symmetrical axis is extracted. At the steps 3401 and 3402, an initial value 
of the processing range is set. The processing range in this case is the vehicle candidate region defined at 

10 the step 3300 of Fig. 5, wherein y coordinate of the upper limit is expressed in Ty; that of the lower limit, in 
By; x coordinate of the left limit, in Lx; and that of the right limit, in Rx. As shown at the step 3403, where 
the differential intensity E (x, y) at the coordinate (x, y) in the image exceeds the threshold Eth, as shown at 
the steps 3404 through 3408, a mid point between the coordinate point and each pixel which is present on 
the same scanning line and has a differential intensity exceeding the threshold Eth is determined, and 

15 added to a histogram corresponding to the mid point position at the step 3406. As shown at the steps 3409, 
3410, 3411 and 3412, the processing is repeated for each edge point within the processing region. Then at 
the step 3413, the peak of histograms thus obtained is determined and stored with the x coordinate at that 
time taken as a symmetrical axis xsym of the vehicle region. Further, in the processing at the steps 3414 
through 3423, the symmetrical region with respect to the symmetrical axis xsym thus determined is 

20 extracted. The processing is such that where after the processing region is initialized at the steps 3414 and 
3415, a pixel (x, y) whose differential intensity exceeds the threshold Eth is confirmed at the step 3416, a 
distance D between the pixel and the symmetrical axis is determined at the step 3417. Tnen at the step 
3418, whether a pixel whose differential intensity exceeds the threshold Eth is present symmetrical axis 
xsym is judged. Where the pixel is present, a pair of the determined symmetrical points are registered at 

25 the step 3419. Further, the processing is performed for each edge point within the processing region as 
shown at the steps 3420 through 3423. Finally, a rectangle circumscribing the symmetrical region is 
determined, and then its width W and height H are determined. Fig. 17a shows results obtained by 
extracting the symmetrical region; and Fig. 17b shows a rectangular region circumscribing the symmetrical 
region of Fig. 17a. 

30 Then at the step 3500 of Fig. 5, with respect to the symmetrical region extracted at the step 3400, an 
initial model of Active Contour Models is set. 

The initial model has a shape approximating a vehicle shape as shown in Fig. 19, and comprises a 
number n of nodes arranged at equal intervals. The figure n of the nodes in this case is preferably about 44. 
The width Wm and height Hm of the initial model are set by multiplying of the W and H determined at the 

35 step 3424 by a parameter P. The parameter P is preferably set to a range 1 .05 to 1 .07. Further, the initial 
model is installed in such a manner that the gravity center Cm of the initial model is matched to the gravity 
center C of the symmetric region determined at the step 3400, whereby the initial value of the Active 
Contour Models can be set to a proper position with respect to the vehicle region. 

At the step 3600 of Fig. 5, the contour of the preceding vehicle is extracted by the use of the Active 

40 Contour Models technique. The dynamic contour mode) is a technique by which an energy function 
Esnakes is defined from the characteristic of an image and the shape of a model, and in the process of 
minimizing the energy function, the contour of an object is extracted. The energy function Esnakes is 
composed of an internal energy Eint as a force relating to the shape of a model such as smoothness and 
internode distance, an image energy Eimage as a force by which the model is drawn to the image 

45 characteristic, and an external energy Econ as a force to restrain externally the change in the shape of the 

model, and is expressed as in the equation (1), wherein vi (i = 1, 2, 3, , n) is a node of the contour 

model. 

Esnakes (vi) = Eint (vi) + Eimage (vi) + Econ (vi) (1) 

50 

Further, the internal energy Eint can be calculated by the equation (2). a and 0 are weight parameters 
for each term. 

Eint (vi) = a |vi - vi-1|2 + 0|vi-1 - 2vi + vi + 1|2 (2) 

55 

The image energy Eimage as a potential field from edges in an image is calculated as a density 
gradient on the image as shown in the equation (3), wherein r is a weight parameter for the image energy. 
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Eimage (vi) = - T j VI (x, y)j (3) 

As the external energy, considering the symmetricalness of the preceding vehicle, there is given a 
restraining force in shape change so that the contour model is contracted symmetrically, as shown in the 
5 equation (4), wherein g is a gravity center coordinate of the contour model; vi* is a symmetrical point of the 
vi with respect to the symmetrical axis passing through the gravity center C; and 5 is a weight parameter for 
the external energy. 

Econ (vi) = 8 | |vi - g| - 1 vi* - g| j (4) 

70 

The energy function Esnakes thus defined is evaluated in the region near each joint of the contour 
model, and the joint is caused to be shifted to a position at which energy becomes the smallest, thereby 
causing the model to be contracted. Where the number of nodes to be shifted becomes a certain threshold 
or less, the contour model is judged to be converged to finish the model contraction. Figs. 22-A and 22-B 

75 show a series of flow of the processing. Then at the steps 3601 and 3602, the parameters are initialized. 
Then at the step 3603, the Esnakes is calculated according to the equations (1) through (4). Then at the 
step 3604, the Esnakes calculated at the step 3603 is compared with the energy of the adjacent pixels. 
Where the Esnakes is judged to be smaller, at the step 3605, the Esnakes is held as the minimum value of 
the energy and at the step 3606, a parameter is added to calculate the energy of the next adjacent pixels, 

20 and then the process returns to the step 3603. On the contrary, where the Esnakes is judged to be larger, 
the process returns to the step 3603 without updating the minimum value of the energy. On the basis of the 
judgment at the step 3607, the processing is repeated for each adjacent regions previously set. Then at the 
step 3608, where the minimum value of the final energy is judged to be obtained at current positions of 
nodes, the process as it is proceeds to the step 3610. On the contrary, where the minimum value of the 

25 energy is judged to be obtained in the adjacent pixels at positions other than the current ones of the nodes, 
at the step 3609, the nodes are shifted to the positions of adjacent pixels, and the process proceeds to the 
step 3610. Fig. 21 shows a concept of the processing at these steps. Then at the step 3610, a parameter is 
added, and then on the basis of the judgment at the step 361 1 , the processing from 3602 to 361 1 is 
repeated for each node. At the step 3611, where the processing for each node is judged to be finished, the 

30 number of shifted nodes at the step 3612 is evaluated, and where the figure is the threshold value or less, 
the contour model is judged to be converged, thereby finishing the processing. On the contrary, where the 
number of shifted nodes is the threshold value or more, the process returns to the step 3601 , at which the 
processing for each node is repeated again. The above-mentioned processing allows the contour of the 
preceding vehicle to be extracted from the road image. 

35 Then at the step 4000 of Fig. 3, a distance to the preceding vehicle recognized at the step 3000 is 
measured. According to the principle of triangulation, the intervehicle distance is calculated by extracting 
the disparity between the preceding vehicles on a pair of stereo images picked up from the video cameras 
1 and 2. 

The flowchart of Fig. 24 shows a series of the flow of the intervehicle distance measuring processing. 
40 First at the step 4100, on the basis of the contour information of the preceding vehicle extracted at the step 
3000 of Fig. 3, a pattern to extract the disparity between stereo images is prepared. 

Figs. 25a and 25b show results obtained by extracting the contour of the preceding vehicle from the 
stereo images of Figs. 4a and 4b, respectively; and Figs. 26a and 26b represent only the contour models 
among the extracted contours. Fig. 27 is a view obtained by reversing the contour model of Fig. 26a with 
45 respect to the symmetric axis of the preceding vehicle extracted at the step 3400 of Fig. 5. The reason why 
the model is caused to be reversed with respect to the symmetric axis is that a distortion between the 
right/left images due to stereo vision is to be corrected. 

Then at the step 4200 of Rg. 24, with the reversed contour model taken as a shift pattern and the 
contour model of Rg. 26b taken as a reference pattern, while establishing a correlation between both the 
so patterns, a shift operation is performed in the scanning direction. Fig. 28 shows a conceptions view of the 
shift operation. As a correlation, the degree of overlapping of contour lines in both the patterns is used. The 
shift by which the correlation becomes the maximum is extracted as a disparity d between both images. 

Then at the step 4300 of Fig. 24, by the use of the disparity extracted at the step 4200, the intervehicle 
distance is calculated according to the equation (6). 
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Intercamera optical axes 

distance DB x focal length f 

Distance D= — — (6) 

5 disparity (pixel) d x pixel size PS 

It is preferable that the distance DB between the optical axes of the video cameras 1 and 2 is about 1 m, 
and that the focal length f is about 7.5 mm. The pixel size PS to be used , which varies with image pick-up 
devices used, has preferably a resolution as high as possible. 

m At the step 5000 of Fig. 3, the distance to the preceding vehicle calculated by the above-mentioned 
processing is outputted. The output of the results is performed by a display installed in the vehicle 
compartment. Fig. 29 shows an example of the output to the display. An output terminal such as RS232C is 
provided to implement various application functions using intervehicle information calculated by this 
apparatus, such as an autocruise for cruising the vehicle with the distance to the preceding vehicle kept 

75 constant, or an intervehicle distance alarm device for alarming the driver if the distance to the preceding 
vehicle becomes a certain value or less. 

Although in the embodiment of the present invention, the Sobel filter has been used in performing edge 
extraction from images, any filter capable of extracting edges from images such as Laplacian may be used. 
Although in the traffic lane region extraction processing, the Hogh transformation has been used in straight- 

20 line approximating the contour point row of white lines, any straight-line approximating technique such as 
the method of least squares may be used. Although in the embodiment of the present invention, the model 
representing the shape of an ordinary automobile has been uesd as the initial model of the dyanmic contour 
model, a model assuming the shape of other vehicle types such as large trucks may be used. 

25 Claims 

1. A vehicle recognition apparatus which recognizes a preceding vehicle from road images inputted by a 
use of vehicle-mounted video cameras and measures a distance to the recognized vehicle, comprising : 

stereo image input means mounted on a vehicle for picking up stereo road scenes in front of the 
30 vehicle; 

image storage means for storing the road scenes information inputted from said stereo image input 
means; 

edge extraction means for applying a differential processing to the road scenes information stored 
in said image storage means to extract edge ; 
35 traffic lane region extraction means for extracting a traffic lane region from the road scenes 

information stored in said image storage means; 

vehicle edge searching means for searching a distribution of the edges extracted by said edge 
extraction means within the traffic lane region extracted by said traffic lane region extraction means , to 
extract the preceding vehicle existing region candidate from the road scenes information stored in said 
40 image storage means; 

symmetric region extraction means for extracting a symmetric region within the preceding vehicle 
existing region candidate defined by said vehicle edge searching means and for limiting further the 
preceding vehicle existing region candidate ; 

initial model setting means for setting an initial contour model for the preceding vehicle existing 
45 region candidate defined by said symmetric region extraction means; 

contour extraction means for extracting contours of the preceding vehicle from the road scenes 
information stored in said image storage means on the basis of the initial values set by said initial 
model setting means; and 

intervehicle distance measuring means for measuring a distance to the preceding vehicle on the 
so basis of the contour of the preceding vehicle extracted by said contour extraction means. 

2. A vehicle recognition apparatus as set forth in claim 1 , comprising; 

traffic lane region extraction means for extracting a self traffic lane region in which self vehicle 
cruises from the road scenes information stored in said image storage means, and then extending 
55 double rightward/leftward a traffic lane width indicated at a lower end of the road scenes by the 
extracted traffic lane region , to obtain right/left approximately adjacent traffic lane regions. 
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m 



A vehicle recognition apparatus as set forth in claim 2, comprising; 

vehicle edge extraction means for extracting a lower end of the preceding vehicle region candidate 
within the self traffic lane region extracted by said traffic lane region extraction means, and then 
extracting right/left side ends of the preceding vehicle region candidate within the adjacent traffic lane 
regions. 

A vehicle recognition apparatus as set forth in claim 3, wherein; 

the vehicle edge extraction means extracts region ends of the vehicle region candidate on the basis 
of dispersion values of a number of edges extracted from a inside of the traffic lane region. 

5. A vehicle recognition apparatus as set forth in claim 1, wherein ; 

the symmetric region extraction means adds up mid-point positions of between edges which is 
present on the same scanning line ,to extract symmetric axes of a whole region. 

75 6. A vehicle recognition apparatus as set forth in claim. 1, wherein ; 

the initial model setting means determines shape and size of model adapted to the preceding 
vehicle existing region candidate defined by said symmetric region extraction means on the basis of 
region width and height of the preceding vehicle existing region candidate. 

20 7. A vehicle recognition apparatus as set forth in claim 6, wherein; 

the initial model setting means matches a gravity center position of the preceding vehicle existing 
region candidate to a gravity center position of the model , to set automatically the model to a proper 
position. 

25 a A vehicle recognition apparatus as set forth in claim 1 , wherein; 

the intervehicle distance measuring means for performing a shift operation in such a manner that 
among the vehicle contours extracted from the right/left stereo road scenes, the contour of one road 
scene is reversed with respect to its symmetric axis and is made a shift pattern, and that of the other 
road scene is made a reference pattern ,and for extracting a shift amount by which a correlation value 

30 of both the patterns becomes minimum , as a disparity between both road scenes , thereby calculating 
the distance to the preceding vehicle. 
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FIG. 1-A 



la 



VIDEO 
CAMERA 1 



ID 



i 



lb 



VIDEO 
CAMERA 2 



2a 



A/D 
CONVERTER 1 



3E 



1 



2b 



A/D 
CONVERTER 2 






in 

CO 

CO 

w 

Q 
Q 
< 

< 

Q 



i 



IMAGE 
PROCESSING 
MPU 



DISPLAY 
CONTROLLER 



3 



8 

Jl 



DISPLAY 



OUTPUT 
INTERFACE 



^-9 



V 



10 



EP 0 626 655 A2 



FIG. 1-B 
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